Medical information processing apparatus and medical information processing method

ABSTRACT

According to one embodiment, a medical information processing apparatus has processing circuitry. The processing circuitry acquires medical data on a subject, acquires numerical data obtained by digitizing an acquisition condition of the medical data, and applies a machine learning model to input data including the numerical data and the medical data, thereby generating output data based on the medical data.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromthe Japanese Patent Application No. 2018-124088, filed Jun. 29, 2018 andthe Japanese Patent Application No. 2019-116625, filed Jun. 24, 2019 theentire contents of all of which are incorporated herein by reference.

FIELD

Embodiments described herein relate generally to a medical informationprocessing apparatus and a medical information processing method.

BACKGROUND

In machine learning using medical data such as medical image data andits raw data, there is a method to apply a deep neural network (DNN)learned from a number of training data in order to restore original datafrom partly deficient medical data. For example, in magnetic resonanceimaging (MRI), there is a method of applying DNN to k-space dataundersampled by cartesian acquisition to generate the k-space data inwhich a deficient part is restored, and obtaining a reconstructed imagebased on the k-space data after restoration. There is also a method ofapplying DNN to undersampled k-space data to directly obtain a restoredimage.

BRIEF DESCRIPTION OF THE DRAWING

FIG. 1 is a view showing a configuration of a magnetic resonance imagingapparatus on which a medical information processing apparatus accordingto the present embodiment is mounted.

FIG. 2 is a drawing showing a typical flow of MR imaging by a processingcircuitry of FIG. 1.

FIG. 3 is a drawing schematically showing a process of generating maskdata in Steps S2 and S3 of FIG. 2.

FIG. 4 is a drawing schematically showing a flow of the first DNNreconstruction according to the present embodiment.

FIG. 5 is a drawing schematically showing a flow of the second DNNreconstruction according to the present embodiment.

FIG. 6 is a drawing schematically showing a flow of a third DNNreconstruction according to the present embodiment.

FIG. 7 is a drawing showing a configuration of a model learningapparatus according to the present embodiment.

FIG. 8 is a drawing schematically showing processing performed by atraining sample generating function of FIG. 7.

FIG. 9 is a view showing another configuration of the medicalinformation processing apparatus according to the present embodiment.

FIG. 10 is a drawing schematically showing numerical data according tothe present embodiment.

FIG. 11 is a drawing schematically showing an input of the numericaldata of FIG. 10 to a machine learning model.

FIG. 12 is a drawing schematically showing an input of another numericaldata to the machine learning model.

FIG. 13 is a drawing schematically showing an input of another numericaldata to the machine learning model.

FIG. 14 is a drawing schematically showing another numerical data.

FIG. 15 is a drawing schematically showing a network structure of themachine learning model according to the present embodiment.

FIG. 16 is a drawing schematically showing a network structure ofanother machine learning model according to the present embodiment.

FIG. 17 is a drawing schematically showing input and output of a machinelearning model having a plurality of CNNs.

DETAILED DESCRIPTION

In general, according to one embodiment, a medical informationprocessing apparatus has processing circuitry. The processing circuitryacquires medical data on a subject, acquires numerical data obtained bydigitizing an acquisition condition of the medical data, and applies amachine learning model to input data including the numerical data andthe medical data, thereby generating output data based on the medicaldata.

Hereinafter, a medical information processing apparatus and a medicalinformation processing method according to the present embodiment willbe described with reference to the drawings.

The medical information processing apparatus according to the presentembodiment is an apparatus in which a processing circuitry thatprocesses medical information is mounted. The medical informationprocessing apparatus according to the present embodiment is realized,for example, by a computer mounted on a medical image diagnosticapparatus. The medical image diagnostic apparatus according to thepresent embodiment may be a single modality apparatus such as a magneticresonance imaging apparatus (MRI apparatus), an X-ray computedtomography apparatus (CT apparatus), an X-ray diagnostic apparatus, aPET (Positron Emission Tomography) apparatus, a single photon emissionCT apparatus (SPECT apparatus), and an ultrasonic diagnostic apparatus,and also may be a combined modality apparatus such as a PET/CTapparatus, a SPECT/CT apparatus, a PET/MRI apparatus, and a SPECT/MRIapparatus. As other examples, the medical information processingapparatus according to the present embodiment may be a computercommunicably connected to the medical image diagnostic apparatus via acable, a network, or the like, or may be a computer independent of themedical image diagnostic apparatus. Hereinafter, the medical informationprocessing apparatus according to the present embodiment is assumed tobe mounted on the magnetic resonance imaging apparatus.

FIG. 1 is a view showing a configuration of a magnetic resonance imagingapparatus 1 on which a medical information processing apparatus 50according to the present embodiment is mounted; As shown in FIG. 1, themagnetic resonance imaging apparatus 1 includes a gantry 11, a couch 13,a gradient field power supply 21, a transmitting circuitry 23, areceiving circuitry 25, a couch motor 27, a sequence control circuitry29, and the medical information processing apparatus 50.

The gantry 11 has a static field magnet 41 and a gradient field coil 43.The static field magnet 41 and the gradient field coil 43 areaccommodated in a housing of the gantry 11. The housing of the gantry 11is formed with a bore having a hollow shape. A transmitting coil 45 anda receiving coil 47 are disposed in the bore of the gantry 11.

The static field magnet 41 has a hollow substantially cylindrical shapeand generates a static magnetic field inside a substantially cylindricalinterior. Examples of the static field magnet 41 used include apermanent magnet, a superconducting magnet or a normal conductingmagnet. Here, a central axis of the static field magnet 41 is defined asa Z axis, an axis perpendicular to the Z axis is defined as a Y axis,and an axis perpendicular to the Z axis is defined as an X axis. The Xaxis, the Y axis and the Z axis constitute an orthogonalthree-dimensional coordinate system.

The gradient field coil 43 is a coil unit attached to the inside of thestatic field magnet 41 and formed in a hollow substantially cylindricalshape. The gradient field gradient field coil 43 receives supply of acurrent from the gradient field power supply 21 to generate a gradientfield. More specifically, the gradient field coil 43 has three coilscorresponding to the X axis, the Y axis, and the Z axis orthogonal toeach other. The three coils form a gradient field in which the magneticfield strength changes along the X axis, the Y axis, and the Z axisrespectively. The gradient fields respectively along the X axis, the Yaxis, and the Z axis are combined to form slice selection gradientfields Gs, phase encoding gradient fields Gp, and frequency encodinggradient fields Gr that are orthogonal to each other in arbitrarydirections. The slice selection gradient fields Gs are used to determinethe imaging cross section arbitrarily. The phase encoding gradientfields Gp are used to change the phase of the MR signal according to thespatial position. The frequency encoding gradient fields Gr are used tochange the frequency of the MR signal according to the spatial position.It should be noted that in the following description, it is assumed thatthe direction of gradient of the slice selection gradient fields Gscorresponds to the Z axis, the direction of gradient of the phaseencoding gradient fields Gp corresponds to the Y axis, and the directionof gradient of the frequency encoding gradient fields Gr corresponds tothe X axis.

The gradient field power supply 21 supplies a current to the gradientfield coil 43 in accordance with a sequence control signal from thesequence control circuitry 29. The gradient field power supply 21supplies a current to the gradient field coil 43 and cause the gradientfield coil 43 to generate a gradient field along each of the X axis, Yaxis, and Z axis. The gradient field is superimposed on the staticmagnetic field formed by the static field magnet 41 and applied to asubject P.

The transmitting coil 45 is disposed, for example, inside the gradientfield coil 43, and receives supply of a current from the transmittingcircuitry 23 to generate a high frequency magnetic field pulse(hereinafter referred to as an RF magnetic field pulse).

The transmitting circuitry 23 supplies a current to the transmittingcoil 45 in order to apply an RF magnetic field pulse for exciting atarget proton in the subject P to the subject P via the transmittingcoil 45. The RF magnetic field pulse oscillates at a resonance frequencyspecific to the target proton to excite the target proton. A magneticresonance signal (hereinafter referred to as an MR signal) is generatedfrom the excited target proton and detected by the receiving coil 47.The transmitting coil 45 is, for example, a whole-body coil (WB coil).The whole-body coil may be used as a transmitting and receiving coil.

The receiving coil 47 receives the MR signal emitted from the targetproton present in the subject P under an action of the RF magnetic fieldpulse. The receiving coil 47 has a plurality of receiving coil elementscapable of receiving the MR signal. The MR signal received is suppliedto the receiving circuitry 25 via wire or wireless. Although not shownin FIG. 1, the receiving coil 47 has a plurality of receiving channelsimplemented in parallel. The receiving channels each include receivingcoil elements that receives the MR signal, an amplifier that amplifiesthe MR signal, and the like. The MR signal is output for each receivingchannel. The total number of the receiving channels and the total numberof the receiving coil elements may be the same, or the total number ofthe receiving channels may be larger or smaller than the total number ofthe receiving coil elements.

The receiving circuitry 25 receives the MR signal generated from theexcited target proton via the receiving coil 47. The receiving circuitry25 processes the MR signal received to generate a digital MR signal. Thedigital MR signal can be expressed in k-space defined by a spatialfrequency. Therefore, hereinafter, the digital MR signal is referred toas k-space data. The k-space data is a type of raw data to be providedfor image reconstruction. The k-space data is supplied to the medicalinformation processing apparatus 50 via wire or wireless.

It should be noted that the transmitting coil 45 and the receiving coil47 described above are merely examples. Instead of the transmitting coil45 and the receiving coil 47, a transmitting and receiving coil having atransmitting function and a receiving function may be used. Also, thetransmitting coil 45, the receiving coil 47, and the transmitting andreceiving coil may be combined.

The couch 13 is installed adjacent to the gantry 11. The couch 13 has atable top 131 and a base 133. The subject P is placed on the table top131. The base 133 slidably supports the table top 131 respectively alongthe X axis, the Y axis, and the Z axis. The couch motor 27 isaccommodated in the base 133. The couch motor 27 moves the table top 131under the control of the sequence control circuitry 29. The couch motor27 may include any motor such as a servo motor or a stepping motor forexample.

The sequence control circuitry 29 has a processor of a CentralProcessing Unit (CPU) or a micro processing unit (MPU) and a memory suchas a read only memory (ROM) or a random access memory (RAM) as hardwareresources. The sequence control circuitry 29 synchronously controls thegradient field power supply 21, the transmitting circuitry 23, and thereceiving circuitry 25 based on the imaging protocol determined by animaging protocol setting function 511 of the processing circuitry 51,executes a pulse sequence corresponding to the imaging protocol to MRimaging the subject P, and acquires the k-space data on the subject P.

As shown in FIG. 1, the medical information processing apparatus 50 is acomputer apparatus having a processing circuitry 51, a memory 52, adisplay 53, an input interface 54, and a communication interface 55.

The processing circuitry 51 includes, as hardware resources, a processorsuch as a CPU, a Graphics Processing Unit (GPU), and a MPU, and a memorysuch as a ROM and a RAM. The processing circuitry 51 functions as thecenter of the magnetic resonance imaging apparatus 1. For example, theprocessing circuitry 51 has the imaging protocol setting function 511, amask data acquisition function 512, a raw data acquisition function 513,an image generating function 514, an image processing function 515, anda display control function 516 by executing various programs.

In the imaging protocol setting function 511, the processing circuitry51 sets an imaging protocol relating to MR imaging of a target by userinstruction via the input interface 54 or automatically. The imagingprotocol is a set of various imaging parameters related to one MRimaging. Examples of applicable imaging parameters according to thepresent embodiment include various imaging parameters set directly orindirectly for performing MR imaging such as imaging time, type ofk-space filling method, type of pulse sequence, TR, TE and the like.

In the mask data acquisition function 512, the processing circuitry 51acquires mask data regarding MR imaging of a target. The mask data isdata in which numerical values corresponding to the number of times ofacquisition and/or the direction of acquisition in target imaging areassigned to a plurality of data acquisition trajectory candidates havinga finite number of elements. The data acquisition trajectory refers toan acquisition trajectory of raw data on k-space and relates to the typeof k-space filling method. For example, if the k-space filling method isa radial method, the data acquisition trajectory is a general line(spoke) passing through the center of k-space. When the k-space fillingmethod is a spiral method or a variable density spiral method, the dataacquisition trajectory is a spiral-shaped curve from the center to theouter periphery of k-space. Acquisition of mask data includes processingsuch as generation of mask data by the processing circuitry 51,selection of any one mask data from among a plurality of mask data,reception, transfer, transmission, and the like of mask data fromanother apparatus.

The raw data acquisition function 513 acquires k-space data. Acquisitionof k-space data includes acquisition of k-space data by MR imagingperformed under the control of the processing circuitry 51, selection ofany one of k-space data from among the plurality of k-space data,reception transfer or transmission of k-space data from otherapparatuses.

In the image generating function 514, the processing circuitry 51performs reconstruction processing using the machine learning model 521on the mask data acquired by the mask data acquisition function 512 andthe k-space data acquired by the raw data acquisition function 513, andgenerates an MR image of the subject P. The machine learning model 521is stored in the memory 52. The machine learning model 521 is aparameterized synthesis function defined by a combination of a pluralityof adjustable functions and parameters (weighting matrices or biases).The machine learning model 521 is realized by a deep network model (DNN:Deep Neural Network) having an input layer, an intermediate layer, andan output layer. Hereinafter, the reconstruction processing using themachine learning model 521 will be referred to as DNN reconstruction.

In the image processing function 515, the processing circuitry 51performs various image processing on the MR image. For example, theprocessing circuitry 51 performs image processing such as volumerendering, surface rendering, pixel value projection processing,Multi-Planer Reconstruction (MPR) processing, Curved MPR (CPR)processing, and the like.

In the display control function 516, the processing circuitry 51displays various information on the display 53. For example, theprocessing circuitry 51 displays the MR image generated by the imagegenerating function 514, the MR image generated by the image processingfunction 515, an imaging protocol setting screen, and the like on thedisplay 53.

The memory 52 is a storage apparatus such as a hard disk drive (HDD), asolid state drive (SSD), an integrated circuitry storage apparatus orthe like that stores various information. Further, the memory 52 may bea drive apparatus or the like that reads and writes various informationfrom and to a portable storage medium such as a CD-ROM drive, a DVDdrive, a flash memory, and the like. For example, the memory 52 storesk-space data, a control program, a machine learning model 521, and thelike.

The display 53 displays various information. For example, the display 53displays an MR image generated by the image generating function 514, anMR image generated by the image processing function 515, a settingscreen of an imaging protocol, and the like. Examples of the display 53that can be used appropriately include a CRT display, a liquid crystaldisplay, an organic EL display, an LED display, a plasma display, or anyother display known in the art.

The input interface 54 includes an input apparatus that receives variouscommands from the user. Examples of the input apparatus that can be usedappropriately include a keyboard, a mouse, various switches, a touchscreen, a touch pad, and the like. It should be noted that the inputapparatus is not limited to those having physical operation parts suchas the mouse and the keyboard. For example, the examples of inputinterface 54 also include an electrical signal processing circuitry thatreceives an electrical signal corresponding to an input operation froman external input apparatus provided separately from the magneticresonance imaging apparatus 1 and outputs the electrical signal receivedto various circuitry.

The communication interface 55 is an interface connecting the magneticresonance imaging apparatus 1 with a workstation, a picture archivingand communication system (PACS), a hospital information system (HIS), aradiology information system (RIS), and the like via a local areanetwork (LAN) or the like. The network IF transmits and receives variousinformation to and from the connected workstation, PACS, HIS and RIS.

It should be noted that the above configuration is merely an example,and the present invention is not limited thereto. For example, thesequence control circuitry 29 may be incorporated into the medicalinformation processing apparatus 50. Also, the sequence controlcircuitry 29 and the processing circuitry 51 may be mounted on the samesubstrate. The sequence control circuitry 29, the gradient field powersupply 21, the transmitting circuitry 23 and the receiving circuitry 25may be mounted on a single control apparatus different from the medicalinformation processing apparatus 50 or may be distributed and mounted ona plurality of apparatuses.

Hereinafter, an operation example of the magnetic resonance imagingapparatus 1 and the medical information processing apparatus 50according to the present embodiment will be described.

FIG. 2 is a drawing showing a typical flow of MR imaging by theprocessing circuitry 51 The process shown in FIG. 2 starts with settingof the imaging protocol of the target MR imaging.

As shown in FIG. 2, the processing circuitry 51 executes an imagingprotocol setting function 511 (Step S1). In Step S1, the processingcircuitry 51 sets an imaging protocol related to the subject P. Asimaging parameters included in the imaging protocol, imaging time, typeof k-space filling method, type of pulse sequence, TR, TE and the likeare set. In the present embodiment, the type of k-space filling methodmay be any method such as the radial method, the spiral method, and theCartesian method. However, in order to specifically explain thefollowing description, it is assumed that the k-space filling method isthe radial method.

When Step S1 is performed, the processing circuitry 51 executes the maskdata acquisition function 512 (Steps S2 and S3). In Steps S2 and S3, theprocessing circuitry 51 generates mask data regarding MR imaging of atarget. The mask data is data in which numerical values indicating thata plurality of candidates of the data acquisition trajectories (spokes)having a finite number of elements are to be acquired or not to beacquired in target imaging are assigned. In other words, the mask datais data indicating a trajectory to be acquired among candidatetrajectories having a finite number of elements.

FIG. 3 is a drawing schematically showing the process of generating maskdata 91 in Steps S2 and S3. As shown in FIG. 3, the data acquisitiontrajectory relating to the radial method passes radially atsubstantially the center of the k-space. As shown in the upper part ofFIG. 3, in the MR imaging according to the present embodiment, dataacquisition trajectory to be acquired (hereinafter, referred to astrajectories to be acquired) is selected from a set 70 of candidates ofthe data acquisition trajectories having a predetermined finite numberof elements (hereinafter referred to as candidate trajectories). Thecandidate trajectories are shown by dotted lines in FIG. 3. The numberof the elements and the angle of the candidate trajectories (number) areset in advance. The number of the elements of the candidate trajectoriesis set to a number that can ensure sufficient image quality when dataacquisition is performed for all candidate trajectories having thenumber of the elements. The number of the elements of the candidatetrajectories is preferably finite. In other words, if the number of theelements of the candidate trajectories is not an irrational number, forexample, it may be a large number such as one million. It should benoted that FIG. 3 shows eight candidate trajectories for the sake ofsimplicity of the drawing.

The angle of the candidate trajectories according to the presentembodiment is defined as a reference angle, for example, an angle from 0degree. The angular interval of adjacent candidate trajectories is setto a predetermined angle. Specifically, it is preferable that theangular interval of adjacent candidate trajectories be set atsubstantially equal intervals. Thus, by limiting the number of theelements and the angle, it is possible to also limit the number of theelements and the angle of candidate trajectories of training data ofmachine learning.

The acquisition angle of each candidate trajectory is set to a multipleof the basic angle. In other words, assuming that the acquisition angleof the first data acquisition trajectory is 0 degrees, setting would bethe acquisition angle of the second candidate trajectory=basic angle,the acquisition angle of the third candidate trajectory=“basic angle×2”,and the acquisition angles of the fourth candidate trajectory=“basicangle×3”, . . . and so forth.

The basic angle may be set to a golden angle, for example, an irrationalnumber of approximately 111.25 degrees if the number of the elements islimited. For example, when the number of the elements is 1000, theacquisition angle of the first candidate trajectory is set to 0 degrees,the acquisition angle of the second candidate trajectory is set to thegolden angle, and the acquisition angle of the third candidatetrajectory is set to “the golden angle×2”, the acquisition angle of thefourth candidate trajectory is set to “the golden angle×3”, . . . and soforth up to the 1000th candidate trajectory.

The acquisition angle may be set to a value obtained by dividing 360degrees by the number of the elements. For example, when the number ofthe elements is 1000, the basic angle may be set to 360/1000degrees=0.36 degrees. Also, the basic angle may be set to a multiple of360/1000 degrees. For example, if the multiple is 309, the basic anglewill be set to 360/1000 degrees×309=111.24 degrees. This allows thebasic angle to be substantially equal to the golden angle.

Number for identification (hereinafter referred to as a trajectorynumber) are assigned to the candidate trajectories. The trajectorynumber is used to generate mask data. The rule of trajectory numberassignment may be any rule. For example, numbers may be assigned inorder from candidate trajectories with small or large acquisitionangles, or numbers may be assigned according to the setting order ofcandidate trajectories.

First, the processing circuitry 51 selects a trajectory to be acquiredbased on the imaging time and the candidate trajectory (Step S2). Forexample, the processing circuitry 51 selects a trajectory to be acquiredfrom among candidate trajectories included in the set 70 in accordancewith a predetermined rule (hereinafter referred to as a selection rule)based on the imaging time set in Step S1. Specifically, first, theprocessing circuitry 51 determines the number of trajectories to beacquired in accordance with the imaging time. Next, the processingcircuitry 51 selects the trajectories to be acquired for the determinednumber from the set 70 in accordance with the selection rule. As theselection rule, for example, it is preferable to select so that theangles between the trajectories to be acquired are substantially thesame. For example, as shown in the middle part of FIG. 3, threecandidate trajectories of substantially equal angles among the eightcandidate trajectories, that is, first, fourth and seventh candidatetrajectories are selected as trajectories to be acquired. In FIG. 3, thetrajectory to be acquired is indicated by a solid line.

The trajectory to be acquired is set for each slice or volume or foreach frame. In other words, the trajectory to be acquired is set foreach image. The quality of image by the radial method is typicallyguaranteed by about 800 spokes. However, according to the presentembodiment, it is possible to guarantee the same image quality by about30 to 50 spokes per image. This is because the acquisition trajectoriesaccording to the present embodiment have substantially equal angularinterval.

Next, the processing circuitry 51 generates mask data 91 based on theselected trajectory to be acquired (Step S3). As shown in the lower partof FIG. 3, the mask data 91 is numerical data in which a numerical valueindicating that it is selected (in other words, data is acquired) or anumerical value indicating that it is not selected (in other words, datais not acquired) is the assigned for each of a plurality of candidatetrajectories. For example, “1” is assigned as a numerical valueindicating that it is selected, and “0” is assigned as a numerical valueindicating that it is not selected. The processing circuitry 51determines whether or not each candidate trajectory is selected, assigns“1” when it is selected, and assigns “0” when it is not selected. Thenumerical values of “0” or “1” are arranged in order of the trajectorynumber of the candidate trajectories. For example, in the case of FIG.3, the mask data 91 is, (1, 0 0, 1, 0, 0, 1, 0) because the first,fourth and seventh candidate trajectories are selected, and the second,third, fifth, sixth and eighth candidate trajectories are not selected.

When Step S3 is performed, the processing circuitry 51 executes the rawdata acquisition function 513 (Step S4). In Step S4, the processingcircuitry 51 instructs the sequence control circuitry 29 to execute MRimaging. The sequence control circuitry 29 synchronously controls thegradient field power supply 21, the transmitting circuitry 23 and thereceiving circuitry 25, performs MR imaging on the trajectory to beacquired selected in Step S2, and acquires k-space data on thetrajectory to be acquired. The k-space data acquired is k-space datacorresponding to the trajectory to be acquired represented by the maskdata generated in Step S3. Since the trajectory to be acquired isselected from among candidate trajectories for the number of theelements, the k-space data acquired in Step S4 is typically sparse.

When Step S4 is performed, the processing circuitry 51 executes theimage generating function 514 (Step S5). In Step S5, the processingcircuitry 51 performs DNN reconstruction on the mask data generated inStep S3 and the k-space data acquired in Step S4 to generate an MRimage.

FIG. 4, FIG. 5 and FIG. 6 are drawing schematically showing flows of DNNreconstruction according to the present embodiment. In the first DNNreconstruction shown in FIG. 4, the processing circuitry 51 applies themachine learning model 521-1 to the k-space data RD and the mask dataMD, and generates an MR image RI corresponding to the k-space data RD.The machine learning model 521-1 receives k-space data and mask data,and the parameters are learned so as to output an MR image correspondingto the k-space data. The structure of the machine learning model 521-1is not particularly limited. Artifacts due to signal deficiency includedin the k-space data RD are reduced more in the MR image RI generated bythe machine learning model 521-1 based on the k-space data RD and themask data MD as compared to the MR image generated by an analyticalreconstruction method such as Fourier transformation on the k-space dataRD. This is because the machine learning model 521-1 uses mask data MDindicating a trajectory to be acquired corresponding to the k-space dataRD as an input in addition to the k-space data RD.

It should be noted that the signal deficiency according to the presentembodiment is a concept including any difference between actual k-spacedata and desired k-space data, including sparse. For example, signaldeficiency includes not only sparse but also signal deterioration due tonoise caused by various causes, information deficiency due to conversionfrom a continuous value to a discrete value generated in the process ofA/D conversion, and the like.

In the second DNN reconstruction shown in FIG. 5, the processingcircuitry 51 applies the machine learning model 521-2 to the k-spacedata RD and the mask data MD, and generate a k-space data RDM with thedeficient part of signal included in the k-space data RD restored. Themachine learning model 521-2 receives k-space data and mask data, andthe parameters are learned so as to output de-noised k-space data. Thek-space data RDM generated by the machine learning model 521-2 based onthe k-space data RD and the mask data MD has reduced artifacts due to asignal deficiency included in the k-space data RD compared to thek-space data RD. This is because the machine learning model 521-2 usesmask data MD indicating a trajectory to be acquired corresponding to thek-space data RD as an input in addition to the k-space data RD.

As shown in FIG. 5, when the k-space data RDM is generated, theprocessing circuitry 51 performs Fourier transformation on the k-spacedata RDM to generate an MR image RI corresponding to the k-space data RDor the k-space data RDM. Since the MR image RI is generated by Fouriertransformation of the k-space data RDM in which the deficient part ofsignal is restored, the image quality is improved compared to the MRimage generated by the Fourier transformation of the k-space data RDincluding the signal deficiency.

In the third DNN reconstruction shown in FIG. 6, the processingcircuitry 51 subjects the k-space data RD to Fourier transformation togenerate a provisional MR image RIM corresponding to the k-space dataRD. Since the provisional MR image RIM is a reconstructed imagegenerated by performing Fourier transformation on k-space data RDincluding a signal deficiency, the provisional MR image RIM includesmany signal deficiencies.

As shown in FIG. 6, when the provisional MR image RIM is generated, theprocessing circuitry 51 applies the machine learning model 521-3 to theprovisional MR image RIM and the mask data MD, and the signals includedin the provisional MR image RIM generates the MR image RI in which thedeficient part of signal is restored. The machine learning model 521-3learns the parameters so that the provisional MR image RIM and the maskdata MD, are input and a de-noised MR image is output. The MR image RIgenerated by the machine learning model 521-3 based on the provisionalMR image RIM and the mask data MD has reduced artifacts caused by thesignal deficiency as compared to the provisional MR image RIM. This isbecause the machine learning model 521-3 uses mask data MD indicating atrajectory to be acquired corresponding to the k-space data RD used inthe provisional MR image RIM as an input in addition to the provisionalMR image RIM.

When Step S5 is performed, the processing circuitry 51 executes thedisplay control function 516 (Step S6). In Step S6, the processingcircuitry 51 displays the MR image generated in Step S5 on the display53.

With the procedure described thus far, the MR imaging by the processingcircuitry 51 is completed.

It should be noted that the flow of the process shown in FIG. 2 is anexample, and the present embodiment is not limited thereto. For example,in Step S2, the processing circuitry 51 automatically selects thetrajectory to be acquired based on the imaging time. However, thepresent embodiment is not limited thereto. For example, the processingcircuitry 51 may select a candidate trajectory specified by the user viathe input interface 54 as a trajectory to be acquired. In this case, forexample, the processing circuitry 51 displays a schematic drawing whichgraphically represents selectable candidate trajectories of a limitednumber of the elements in a selectable manner on the display 53. As aschematic drawing, for example, an image in which candidate trajectoriesdisposed in the k-space and their trajectory numbers are drawn as shownin the upper part of FIG. 3 is preferable. The user designates anarbitrary trajectory as the trajectory to be acquired from among thecandidate trajectories included in the displayed schematic drawing viathe input interface 54. The processing circuitry 51 may select thedesignated trajectory as the trajectory to be acquired.

The above embodiment is based on the assumption that one or zero timesof data acquisition is performed for each trajectory to be acquired forMR imaging of one image. However, the present embodiment is not limitedthereto. For example, data acquisition may be performed twice or morefor a certain trajectory to be acquired. By performing data acquisitiontwo or more times, the reliability of k-space data related to thetrajectory to be acquired can be improved. For example, when dataacquisition is performed twice for the first and fourth trajectories,once for the second, fifth and seventh trajectories, and zero time forthe third, sixth and eighth trajectories, mask data will be (2, 1, 0, 2,1, 0, 1, 0).

In the mask data according to the above embodiment, it is assumed that anumerical value corresponding to the number of times of acquisition isassigned to each data acquisition trajectory. However, the presentembodiment is not limited thereto. For example, the numerical value “1”may be uniformly assigned to a data acquisition trajectory in whichacquisition is performed twice or more. In addition, the numerical value“1” may be assigned in the case of 0 times and 2 or more times, and thenumerical value “0” may be assigned in the case of 1 time. Arbitrarynumerical values may be assigned to the mask data according to thepresent embodiment in accordance with other rules.

It should be noted that although the non-negative value is used as themask data in the description of the present embodiment, the presentinvention is not limited thereto, and a negative value, for example, −1may be included.

Also, the numerical value of the mask data is assumed to have a valuecorresponding to the number of times of acquisition. However, thepresent embodiment is not limited thereto. For example, a case where thesame trajectory is acquired in the positive direction and the negativedirection in the radial acquisition is considered. When the dataacquisition trajectory in the positive direction and the dataacquisition trajectory in the negative direction are treated asdifferent trajectories, numerical values corresponding to the number oftimes of acquisition are assigned to each data acquisition trajectory.Without making distinction between data acquisition trajectories in thepositive direction and data acquisition trajectories in the negativedirection, a numerical value according to the acquisition direction or anumerical value according to the combination of the number of times ofacquisition and the direction of acquisition may be assigned to one dataacquisition trajectory. For example, in the case of acquiring 100 timeseach in the positive direction and the negative direction with respectto a 0° data acquisition trajectory, (1, 0) or the like is assigned tothe data acquisition trajectory. It should be noted that “1” in (1, 0)is an example of the numerical value indicating a positive direction,and “0” is an example of a numerical value indicating a negativedirection.

The structure of the machine learning model 521 can be changed in designas appropriate. For example, in the case of the second DNNreconstruction, the machine learning model 521 may incorporate an FFT(Fast Fourier Transfer) layer after an arbitrary multi-layer network(hereinafter, referred to as the present network layer) which inputsk-space data RD and mask data MD and outputs k-space data RDM. Thek-space data is input to the FFT layer, the FFT is applied to the inputk-space data, and an MR image is output. Accordingly, the machinelearning model 521 alone can output the MR image RI based on the k-spacedata RD and the mask data MD.

The machine learning model 521 may also have a chain structure in whichunit network structures including an Inverse Fast Fourier Transfer(IFFT) layer following the present network layer and the FFT layer arecascaded. An MR image is input to the IFFT layer, IFFT is applied to theinput MR image, and k-space data is output. The chain structure canimprove the restoration accuracy of the deficient part of signal.

In the chain structure, a matching layer may be provided after the IFFTlayer. K-space data based on the MR image output from the presentnetwork layer and k-space data before processing input to the presentnetwork layer are input to the matching layer, and matching processingis performed on the completed k-space data using the k-space data beforeprocessing, and the k-space data after the matching processing iscompleted is output. The k-space data completed in the matchingprocessing, the k-space data before processing is weighted and added foreach pixel according to the degree of signal deficiency. For example,the lower the signal deficiency degree, the higher the weight given tothe pixel value of the k-space data before processing, and the higherthe signal deficiency degree, the lower the weight is given to the pixelvalue of the k-space data before processing. Accordingly, theconsistency between the processed k-space data and the k-space databefore processing can be secured.

Also, in the machine learning model 521, an element product operationlayer may be provided in the previous stage of the present networklayer. The element product operation layer calculates an element productof k-space data and mask data, and outputs element product data. Thepresent network layer receives k-space data and element product data,and outputs MR image data in which a deficient part of signal of thek-space data is restored. The unit network structure of the presentnetwork layer and the element product operation layer may be cascaded.

In the above description, the k-space data input to the machine learningmodel 521 is the original k-space data acquired by MR imaging. However,the present embodiment is not limited thereto. The k-space dataaccording to the present embodiment may be computational k-space datagenerated by performing forward projection processing on an MR image.Further, the k-space data or MR image according to the presentembodiment may be raw data subjected to any signal processing such assignal compression processing, resolution decomposition processing,signal interpolation processing, resolution synthesis processing and thelike. The raw data according to the present embodiment may be hybriddata generated by performing Fourier transformation on k-space data inthe readout direction.

Next, learning of the machine learning model 521 will be described. Themachine learning model 521 is generated by a model learning apparatus.The model learning apparatus causes the machine learning model toperform machine learning according to a model learning program based ontraining data including a plurality of training samples, and generates alearned machine learning model (hereinafter referred to as a learnedmodel). The model learning apparatus is a computer such as a workstationhaving a processor such as a CPU and a GPU. The model learning apparatusand the medical information processing apparatus 50 may or may not becommunicably connected via a cable or a communication network. Inaddition, the model learning apparatus and the medical informationprocessing apparatus 50 may be configured by an integrated computer.

FIG. 7 is a drawing showing the configuration of a model learningapparatus 6 according to the present embodiment. As shown in FIG. 7, themodel learning apparatus 6 includes a processing circuitry 61, a memory62, an input interface 63, a communication interface 64, and a display65.

The processing circuitry 61 includes a processor such as a CPU or a GPU.The processor executes a training sample generating function 611, atraining function 612, a display control function 613 and the like byactivating a model learning program installed in the memory 62 and thelike. It should be noted that the respective functions 611 to 613 arenot limited to being realized by a single processing circuitry. Aplurality of independent processors may be combined to constitute aprocessing circuitry, and each processor may execute a program torealize each of the functions 611 to 613.

In the training sample generating function 611, the processing circuitry61 generates a training sample which is a combination of input data andoutput data. The processing circuitry 61 generates training samplesbased on k-space data acquired along all candidate trajectories.

In the training function 612, the processing circuitry 61 causes themachine learning model to train parameters based on training data on aplurality of training samples. By training parameters by the trainingfunction 612, a learned machine learning model 521 shown in FIG. 1 isgenerated.

In the display control function 613, the processing circuitry 61displays the training data, the learning result and the like on thedisplay 65.

The memory 62 is a storage apparatus such as a ROM, a RAM, an HDD, anSSD, an integrated circuitry storage apparatus, and the like for storingvarious information. The memory 62 stores, for example, a model learningprogram for learning a multi-layered network. The memory 62 may be adrive apparatus for reading and writing various information from/to aportable storage medium such as a CD, a DVD, a flash memory, or asemiconductor memory device such as a RAM, in addition to the storageapparatus. Also, the memory 62 may be in another computer Connected tothe model learning apparatus 6 via a network.

The input interface 63 receives various input operations from the user,converts the received input operations into electrical signals, andoutputs the electric signals to the processing circuitry 61.Specifically, the input interface 63 is connected to input apparatusessuch as a mouse, a keyboard, a trackball, a switch, a button, ajoystick, a touch pad, and a touch panel display. The input interface 63outputs an electrical signal corresponding to the input operation to theinput apparatus to the processing circuitry 61. The input apparatusconnected to the input interface 63 may be an input apparatus providedin another computer connected via a network or the like.

The communication interface 64 is an interface for performing datacommunication with the medical information processing apparatus 50, themedical image diagnostic apparatus, and another computer.

The display 65 displays various information in accordance with thedisplay control function 613 of the processing circuitry 61. Forexample, the display 65 displays training data, learning results, andthe like. The display 65 also outputs a GUI or the like for receivingvarious operations from the user. For example, as the display 65, aliquid crystal display, a CRT display, an organic EL display, a plasmadisplay, or any other display can be appropriately used.

Next, an operation example of the model learning apparatus 6 accordingto the present embodiment will be described.

As described above, the processing circuitry 61 generates trainingsamples based on k-space data acquired along all candidate trajectories.The types of input data and output data differ depending on the type ofmachine learning model 521 shown in FIG. 4, FIG. 5 and FIG. 6. In thecase of the machine learning model 521-1 of FIG. 4, mask data andk-space data including signal deficiency are used as input data, and anMR image with reduced signal deficiency is used as output data. In thecase of the machine learning model 521-2 of FIG. 5, mask data andk-space data including signal deficiency are used as input data, andk-space data with reduced signal deficiency is used as output data. Inthe case of the machine learning model 521-3 of FIG. 6, mask data and MRimage including signal deficiency are used as input data, and an MRimage with reduced signal deficiency is used as output data.

Hereinafter, as an example of the machine learning model 521, machinelearning processing will be described with an example of the machinelearning model 521-1 in FIG. 4 in which parameters are learned so as tooutput an MR image by inputting k-space data and mask data.

FIG. 8 is a drawing schematically showing processing executed by thetraining sample generating function 611. As shown in FIG. 8, fullk-space data RDF is acquired by the processing circuitry 61. The fullk-space data is k-space data obtained by acquiring data for allcandidate trajectories of a finite number of elements. The full k-spacedata RDF is acquired in advance by the magnetic resonance imagingapparatus 1 or the like.

As shown in FIG. 8, in the training sample generating function 611, theprocessing circuitry 61 performs under-sampling processing on the fullk-space data RDF, and generates virtual k-space data RD1 to RDn for allcombinations of processing target trajectories (hereinafter referred toas trajectory combinations). For example, the processing circuitry 61calculates k-space data corresponding to each trajectory combination bysimulation based on the full k-space data RDF. In FIG. 8, virtualk-space data RD1 to RDn illustrate k-space data relating to threeprocessing target trajectories, but the number of processing targettrajectories constituting virtual k-space data RD1 to RDn may be anynumber as long as it is smaller than the number of candidatetrajectories constituting the full k-space data RDF. The virtual k-spacedata RD1 to RDn are used as input data of the machine learning model521-1.

When under-sampling processing is performed, the processing circuitry 61performs Fourier transformation on each of the virtual k-space data RD1to RDn to generate virtual MR images RI1 to RIn corresponding to thevirtual k-space data RD1 to RDn. The virtual MR images RI1 to RIn areused as correct MR images of the machine learning model 521-1.

The processing circuitry 61 performs mask data conversion processing oneach of the virtual k-space data RD1 to RDn, and generates mask data MD1to MDn corresponding to the virtual k-space data RD1 to RDn.Specifically, the processing circuitry 61 first specifies the trajectorycombination of each of the virtual k-space data RD1 to RDn. Then, foreach candidate trajectory, the processing circuitry 61 assigns anumerical value “1” to the trajectory included in the trajectorycombination and assigns a numerical value “0” to a trajectory notincluded in the trajectory combination. Thereby, mask data MD1 to MDnare generated. The mask data MD1 to MDn are used as input data of themachine learning model 521-1.

The mask data MD1 to MDn, the k-space data RD1 to RDn, and the MR imagesRI1 to RIn are associated with each other for each trajectorycombination. The mask data MD1 to MDn, the k-space data RD1 to RDn, andthe MR images RI1 to RIn for each trajectory combination are treated astraining samples. The training samples for each trajectory combinationare stored in the memory 62.

Accordingly, the processing by the training sample generating function611 ends. Next, training processing by the training function 612 isperformed. In the training function 612, the processing circuitry 61applies k-space data RD1 to RDn and mask data MD1 to MDn to the machinelearning model to perform forward propagation processing for eachtrajectory combination, and outputs MR image (hereinafter referred to asestimated MR image). Next, the processing circuitry 61 applies thedifference (error) between an estimated MR result and the correct MRimage to the machine learning model to perform back propagationprocessing, and calculates a gradient vector. Next, the processingcircuitry 61 updates parameters such as a weighting matrix and a bias ofthe machine learning model based on the gradient vector. By repeatingthe forward propagation processing, the back propagation processing, andparameter updating processing while changing the training sample, alearned machine learning model 521-1 is generated.

The learned machine learning model 521-1 is stored in the memory 62.Also, the learned machine learning model 521-1 is transmitted to themagnetic resonance imaging apparatus 1 via the communication interface64.

As described above, the type of the k-space filling method according tothe present embodiment may be a spiral method. Also in this case, maskdata can be generated as in the radial method. The candidate trajectoryrelating to the spiral method draws a spiral starting from the center ofk-space and ending at an arbitrary point. Similar to the radial method,the processing circuitry 51 selects an arbitrary candidate trajectoryfrom among a set including a plurality of candidate trajectories as atrajectory to be acquired, and generates mask data corresponding to theselected trajectory to be acquired. It should be noted that since thecandidate trajectories that can be generated depend on hardwareperformance of the gradient coil, it is necessary to prepare trainingdata for each target hardware. Alternatively, a set of candidatetrajectories of the spiral method may be defined in accordance with lowgradient coils so as to be compatible with a plurality of hardware.

In the above description, the medical information processing apparatus50 is incorporated in the magnetic resonance imaging apparatus 1.However, the present embodiment is not limited thereto.

FIG. 9 is a view showing another configuration of the medicalinformation processing apparatus 50 according to the present embodiment.As shown in FIG. 9, the medical information processing apparatus 50 is asingle computer independent of a modality apparatus such as a magneticresonance imaging apparatus. Similar to the medical informationprocessing apparatus 50 of FIG. 1, the medical information processingapparatus 50 of FIG. 9 includes a processing circuitry 51, a memory 52,a display 53, an input interface 54, and a communication interface 55.The functions and the like of each configuration are the same as thosein FIG. 1.

It should be noted that the medical information processing apparatus 50of FIG. 9 can process raw data acquired by any modality apparatus. Forexample, the raw data may be sinogram data acquired by an x-ray computedtomography apparatus.

As described above, the medical information processing apparatus 50according to the present embodiment has the processing circuitry 51. Theprocessing circuitry 51 implements at least a mask data acquisitionfunction 512, a raw data acquisition function 513, and an imagegenerating function 514. In the mask data acquisition function 512, theprocessing circuitry 51 acquires mask data in which numerical valuescorresponding to the number of times of acquisition and/or the directionof acquisition in target imaging are assigned to a plurality of dataacquisition trajectory candidates having a finite number of elements. Inthe raw data acquisition function 513, the processing circuitry 51acquires the raw data acquired in the target imaging on the subject P.In the image generating function 514, the processing circuitry 51performs reconstruction processing using the machine learning model 521on the acquired mask data and the acquired raw data to generate amedical image on the subject P.

According to the above configuration, the machine learning model 521uses, in addition to the k-space data, mask data indicating a trajectoryto be acquired corresponding to the k-space data as an input. Thus, theimage quality of the output medical image can be improved as compared toa case where only the k-space data is input. Further, since thecandidate trajectories are limited to trajectories having apredetermined finite number of elements, it is not necessary tocalculate the trajectories to be acquired for each MR imaging. Inrelation to this, it is possible to easily match the data acquisitiontrajectory used for training data and the data acquisition trajectoryused for MR imaging.

In the above embodiment, the input of the machine learning model ismedical data such as raw data or medical image and mask data to whichnumerical values according to the number of times of acquisition and/orthe direction of acquisition in target imaging for a plurality of dataacquisition trajectory candidates having a finite number of elements areassigned. However, the present embodiment is not limited thereto. Theconcept of mask data can be extended to numerical data obtained bydigitizing acquisition condition for medical data. The acquisitioncondition for medical data is a concept including not only an imagingprotocol for medical imaging but also data processing conditions for rawdata and image processing conditions for medical images. The imagingprotocol for medical imaging includes not only the data acquisitiontrajectory described above, but also the type of pulse sequence, a framenumber, and a type of k-space filling trajectory.

In the above embodiment, the machine learning model is used for DNNreconstruction from raw data to medical images. However, the presentembodiment is not limited thereto. The output of the machine learningmodel may be any data as long as it is data used for medical diagnosis.For example, a machine learning model may be configured to outputresults of identification such as restoration of a deficient part of rawdata, restoration of a deficient part of a medical image, generation ofa segmentation image in which an anatomical tissue or the like issegmented, and the anatomical tissue.

FIG. 10 is a drawing schematically showing numerical data 93; Numericaldata 93 shown in FIG. 10 is data obtained by digitizing the type ofpulse sequence. As shown in FIG. 10, the numerical data 93 includeselements 94 for two or more finite number of elements. The number of theelements matches the number of a plurality of candidate conditionsincluded in the category to which the acquisition condition to bedigitized belongs. For example, as shown in FIG. 10, when the categoryof the acquisition condition to be digitized is the type of pulsesequence, examples of candidate conditions include three types; FSE(Fast Spin Echo), FE (Field Echo) and EPI (Echo Planar Imaging). In thiscase, the numerical data 93 is expressed as (FSE, FE, EPI).

Each element 94 corresponds to a numerical value corresponding towhether or not each of a plurality of candidate conditions is adopted bythe acquisition condition of the target imaging. For example, as thevalue of the element 94, one of a first value indicating that thecandidate condition in question is adopted and a second value indicatingthat the candidate condition in question is not adopted is assigned. Inthis case, the numerical data 93 is referred to as one-hot vector. Anyinteger or natural number may be used as long as the first value and thesecond value are different values. For example, the first value is setto “1” and the second value is set to “0”.

For example, as shown in FIG. 10, when the acquisition condition of thetarget imaging adopts FSE is adopted, the value of the element 84corresponding to FSE is set to “1”, the value of the element 84corresponding to FE is set to “0”, and the value of the element 84corresponding to EPI is set to “0”. It should be noted that the numberof the elements may be any number of finite or infinite. In other words,the number of candidate conditions may be any number. For example, it ispossible to set the number of the elements, that is, the number ofcandidate conditions to one million or more.

FIG. 11 is a drawing schematically showing an input of the numericaldata 93-1 of FIG. 10 to a machine learning model. As shown in the upperpart of FIG. 11, when FSE is adopted, the value of the elementcorresponding to FSE of numerical data 93-1 is “1”, the value of theelement corresponding to FE is “0”, and EPI corresponds. The value ofthe element is set to “0”. A combination of the numerical data 93-1 andthe input medical image 101 is input to the machine learning model. Asshown in the middle part of FIG. 11, when FE is adopted, the value ofthe element corresponding to FE of numerical data 93-1 is set to “0”,the value of the element corresponding to FE is set to “1”, and thevalue of the element corresponding to EPI is set to “0”. A combinationof the numerical data 93-1 and the input medical image 101 is input tothe machine learning model. As shown in the lower part of FIG. 11, whenEPI is adopted, the value of the element corresponding to EPI ofnumerical data 93-1 is “0”, the value of the element corresponding to FEis “0”, and the value of the element corresponding to EPI is set to “1”.A combination of the numerical data 93-1 and the input medical image 101is input to the machine learning model. By adding the numerical data93-1 obtained by digitizing the type of pulse sequence in this manner tothe input of the machine learning model, it becomes possible to generateoutput data taking the type of pulse sequence into consideration as wellas the medical image or raw data. Therefore, the accuracy of the outputdata of the machine learning model is improved.

FIG. 12 is a drawing schematically showing the input of anothernumerical data 93-2 to the machine learning model. Numerical data 93-2shown in FIG. 12 is a digitized acquisition condition “golden angleframe number”. The golden angle frame number is a frame number ofdynamic imaging using the radial method in which the acquisition angleof the data acquisition trajectory is set according to the rule of thegolden angle, and the frame number is associated with the golden angleof the reference data acquisition trajectory relating the frame. Thereference data acquisition trajectory is a data acquisition trajectoryof any order, such as first, last, or intermediate, when the frame isconstructed of a plurality of data acquisition trajectories. In thiscase, the numerical data 93-2 is expressed as (golden angle frame #1,golden angle frame #2, . . . ). The number of the elements of thenumerical data 93-2 matches or corresponds to the number of frame numbercandidates.

As shown in the upper part of FIG. 12, when the input medical image 101is the golden angle frame #1, the value of the element corresponding tothe golden angle frame #1 of the numerical data is “1”, and the value ofthe element corresponding to other frames is set to “0”. A combinationof the numerical data 93-2 and the input medical image 101 is input tothe machine learning model. As shown in the middle part of FIG. 12, whenthe input medical image 101 is the golden angle frame #2, the value ofthe element corresponding to the golden angle frame #2 of the numericaldata is “1”, and the value of the element corresponding to other framesis set to “0”. A combination of the numerical data 93-2 and the inputmedical image 101 is input to the machine learning model. By adding thenumerical data obtained by digitizing the golden angle frame number inthis manner to the input of the machine learning model, it becomespossible to generate output data taking the golden angle frame numberinto consideration as well as the medical image or raw data. Therefore,the accuracy of the output data of the machine learning model isimproved.

It should be noted that the acquisition angle and acquisition order ofdata acquisition trajectories of the radial method in dynamic imagingare not limited to those using the golden angle, but may be those usingan inverted bit pattern or those using a sequential pattern. In thiscase, the number of the elements of the numerical data matches orcorresponds to the number of candidates for the inverted bit pattern orthe sequential pattern. The value of each element is set according towhether or not the inverted bit pattern or the sequential pattern forthe input medical image adopts the inverted bit pattern or thesequential pattern corresponding to the element.

FIG. 13 is a drawing schematically showing the input of anothernumerical data 93-3 to the machine learning model. The numerical data93-3 shown in FIG. 13 is obtained by digitizing acquisition condition“pseudo random pattern number”. The pseudo random pattern number is anumber of a random number pattern of dynamic imaging using the pseudorandom Cartesian method, and the frame number is associated with therandom number pattern related to the frame. It should be noted that thepseudo random Cartesian method refers to the Cartesian method in which adata acquisition trajectory is set according to a random number pattern.In this case, the numerical data 93-3 is expressed as (pseudo randompattern #1, pseudo random pattern #2, . . . ). The number of theelements of the numerical data 93-3 matches or corresponds to the numberof candidates for the random number pattern.

As shown in the upper part of FIG. 13, when the input medical image 101is the pseudo random pattern #1, the value of the element correspondingto the pseudo random pattern #1 of the numerical data 93-3 is “1”, andthe values of the elements corresponding to other pseudo random patternsare set to “0”. A combination of the numerical data 93-3 and the inputmedical image 101 is input to the machine learning model. As shown inthe middle part of FIG. 13, when the input medical image 101 is thepseudo random pattern #2, the value of the element corresponding to thepseudo random pattern #2 of the numerical data 93-3 is “1”, and theelements corresponding to other pseudo random patterns are set to “0”. Acombination of the numerical data 93-3 and the input medical image 101is input to the machine learning model. By adding the numerical dataobtained by digitizing the pseudo random pattern number in this mannerto the input of the machine learning model, it becomes possible togenerate output data taking the golden angle frame number intoconsideration as well as the medical image or raw data. Therefore, theaccuracy of the output data of the machine learning model is improved.

The acquisition conditions for dynamic (time-series) imaging, which areto be digitized, are not limited to those described above, and may be,for example, frame numbers defined by the cardiac phase. The framenumber defined by the cardiac phase is, for example, the number of thecardiac phase of the frame when the R wave to the next R wave is 100%.

The acquisition condition to be digitized may be an acceleration factorof parallel imaging. In this case, the number of the elements ofnumerical data is defined as the number of candidates for theacceleration factor rate. The value of the element has a value accordingto whether or not the acceleration factor corresponding to the elementis adopted in the target imaging. The artifacts resulting from parallelimaging have features according to the acceleration factor. Therefore,the machine learning model can detect the feature according to theacceleration factor by adding numerical data obtained by digitizing theacceleration factor of parallel imaging to the input of the machinelearning model, and thus the accuracy of the output data of the machinelearning model is enhanced.

The acquisition conditions to be digitized are not limited to the above,and may be, for example, data processing conditions for raw data orimage processing conditions for medical images. As the data processingconditions or the image processing conditions, for example, there is thenumber of times of repetition of the repeat operation. The repeatoperation is an operation such as noise removal processing, edgeemphasis processing, smoothing processing and the like that isrepeatedly applied to raw data or medical images, and the number oftimes of repetition is the number obtained by repeating the operation.Alternatively, the repeat operation is an operation for reducing thereconstruction error between the raw data and the reconstructed imagewhen reconstructing the medical image from raw data, and the number oftimes of repetition is the number of times of repeating the operationwhen the operation is performed alternately. The operation is performedusing a filter, DNN or the like. The number of the elements of thenumerical data is defined as the number of candidates for the number oftimes of repetition, and the value of the element has a valuecorresponding to whether or not the target operation is repeated by thenumber of times of repetition corresponding to the element. As describedabove, by adding medical data after the repeat operations and numericaldata obtained by digitizing the number of times of repetition ofrepeated operation to the input of the machine learning model, it ispossible to generate output data considering not only the medical databut also the number of times of repetition. Therefore, the accuracy ofthe output data of the machine learning model is improved. It should benoted that the number of times of repetition may be a one-hot vector.Alternatively, the number of times of repetition may be a vector givenin the form of a one-hot vector for the first few times (for example,three times) and given as the number of times of repetition beyond thatnumber of times.

Numerical data relating to data processing conditions or imageprocessing conditions may be digitized data which expresses whether ornot a plurality of types of data processing or image processing areadopted. In this case, the number of the elements of numerical data isthe number of candidates for data processing or image processing, andthe value of each element is set to a value according to whether or notindividual data processing or image processing is adopted. For example,numerical data may be expressed as (noise reduction, edge emphasis,segment processing) or the like.

The number of the elements of the above numerical data is assumed tomatch the number of candidate conditions. However, the presentembodiment is not limited thereto.

FIG. 14 is a view schematically showing another numerical data 93-4. Asshown in FIG. 14, the numerical data 93-4 includes an element 94corresponding to any of the candidate conditions and an element 95corresponding to the fact that none of the candidate conditions is met.For example, the value of the element 95 is set to “0” when itcorresponds to any of the candidate conditions, and is set to “1” whenit corresponds to any of the candidate conditions. For example, as shownin FIG. 14, when the numerical data 93-4 is data obtained by digitizingthe pulse sequence and the pulse sequence of the actual MR imaging isnot any of FSE, FE, and EPI, the value of the element 95 is set to “1”.By providing the element 95 in the numerical data 93-4, it is possibleto clearly indicate to the machine learning model that the pulsesequence of the actual MR imaging does not correspond to any of thecandidate conditions.

It is assumed that the above numerical data is obtained by digitizingone type of acquisition condition such as the type of pulse sequence andframe number. However, the present embodiment is not limited thereto.For example, numerical data may represent the type of pulse sequence andthe frame number as a vector of one column. In this case, the numericaldata is specifically represented by (FSE, FE, EPI, frame number 1, framenumber 2, . . . ) or the like.

Further, in the above description, it was assumed that the element ofthe numerical data was a one-hot vector representing whether or not thecandidate condition in question is adopted in binary. However, theelement may be a vector that represents multi-level concepts such as thestrength, direction, and number of times of the candidate condition inquestion in three or more values.

Next, the network structure of the machine learning model according tothe present embodiment will be described.

FIG. 15 is a view schematically showing a network structure of a machinelearning model 521-4 according to the present embodiment. As shown inFIG. 15, it is assumed as an example that the machine learning model521-4 is a DNN in which parameters has been trained to output a medicalimage having no noise, for example by inputting a combination of amedical image #1 having noise and numerical data #2 obtained bydigitizing the acquisition condition for the medical image. It isassumed that there is one set each of the medical image #1 and thenumerical data #2.

The machine learning model 521-4 has a CNN layer 523. The CNN layer 523is configured to multiply each of the medical image #1 and the numericaldata #2 by a weight different for each channel and add the product. Forexample, a plurality of filters with the number of channel ch areapplied to medical image #1, and a plurality of filters with the samenumber of channels ch are applied to numerical data #2. The medicalimage and the numerical data after filtration are added for each channelby the adder 203 and converted into addition data. The addition data isconverted into bias data by adding a constant value for each channel bythe bias 204. Bias data for each channel is output from the CNN layer523.

The network structure of the CNN layer 523 is an example, and variousmodifications are possible. For example, the CNN layer 523 may beprovided with multistage filters 201 and 202 instead of the single-stagefilters 201 and 202. For example, the size of the filter 201 may be setto 5×5, and the size of the filter 202 may be set to 1×1 or the like,but the size of the filters 201 and 202 may have any size.

Following the CNN layer 523 is a postposition layer 525. Thepostposition layer 525 performs an operation on the bias data for aplurality of channels from the CNN layer 523 and outputs a medical imagehaving no noise as an output of the machine learning model 521-4. Thepostposition layer 525 has at least one or more fully connecting layersand an output layer, but in addition to these layers, it may have anylayer such as one or more CNN layers, a pooling layer, and anormalization layer.

The input of the CNN layer 523 shown in FIG. 15 was assumed to be oneset of medical image and one set of numerical data. However, the presentembodiment is not limited thereto. The input of the CNN layer may bemultiple sets of medical images and the same set of numerical data.

FIG. 16 is a drawing schematically showing a network structure ofanother machine learning model 521-5 according to the presentembodiment. As shown in FIG. 16, it is assumed as an example that themachine learning model 521-5 is a DNN in which parameters has beentrained to output a medical image having no noise, for example, wheninputting a combination of a medical image #1 having noise, a medicalimage #2 having noise, numerical data #3 obtained by digitizingacquisition conditions relating to medical image #1 and numerical data#4 obtained by digitizing the acquisition condition relating to themedical image #2. The medical image #2 is, for example, a medical imagederived from the medical image #1, such as a copy of the medical image#1 or a medical image obtained by performing arbitrary image processingon the medical image #1. The medical image 2 may be another medicalimage acquired under the same acquisition condition as that for themedical image #1, or may be a medical image acquired under anotheracquisition condition.

The machine learning model 521-5 has a CNN layer 527. The CNN layer 527is configured to multiply the medical image #1, the medical image #2,the numerical data #3 and the numerical data #4 respectively by weightsdifferent for respective channels and add respective products. Forexample, a plurality of filters 211 by the number of channels ch areapplied to medical image #1, a plurality of filters 212 by the samenumber ch are applied to medical image #2, a plurality of filters 213 ofsame number ch are applied to numerical data #3, and a plurality offilters 214 of the same number ch are applied to numerical data #4. Themedical images and the numerical data after filtration are added foreach channel by the adder 215 and converted into addition data. Theaddition data are converted into bias data by the bias 216 by beingadded with a constant value for the respective channels. Bias data forthe respective channels are output from the CNN layer 527.

The network structure of the CNN layer 527 is an example, and variousmodifications are possible. For example, the CNN layer 527 may beprovided with multistage filters 211, 212, 213, 214 instead of thesingle-stage filters 211, 212, 213, 214.

Following the CNN layer 527 is a postposition layer 529. Thepostposition layer 529 performs an operation on the bias data for aplurality of channels from the CNN layer 527 and outputs a medical imagehaving no noise as an output of the machine learning model 521-5. Theoutput medical image is a medical image corresponding to the medicalimage #1, such as an image obtained by removing noise from the medicalimage #1. The postposition layer 529 has at least one or more fullyconnecting layers and an output layer, but in addition to these layers,it may have any layer such as one or more CNN layers, a pooling layer,and a normalization layer.

It should be noted that in the case of a machine learning model having aplurality of CNN layers, numerical data need not necessarily be input toeach CNN layer.

FIG. 17 is a drawing schematically showing input and output of a machinelearning model 521-6 having a plurality of CNN layers 531. The machinelearning model 521-6 shown in FIG. 17 has, as an example, a series ofthree layers; a CNN layer 531-1, a CNN layer 531-2 and a CNN layer531-3. The postposition layer 533 is provided after the CNN 531-3. TheCNN 531-1 receives an input medical image, the CNN layer 531-2 receivesoutput data from the CNN layer 531-1, and the CNN layer 531-3 receivesoutput data from the CNN layer 531-2. The postposition layer 533receives the output data from the CNN layer 531-3, and outputs, forexample, a medical image in which noise is removed from the inputmedical image #1 as an output of the machine learning model 521-6. Thenetwork structures of the CNN layer 531-1, the CNN layer 531-2 and theCNN layer 531-3 may be identical or different, and may be designedarbitrarily.

As shown in FIG. 17, numerical data #2 may not be input for all threelayers; the CNN layer 531-1, the CNN layer 531-2 and the CNN layer531-3. For example, the numerical data #2 may be input not to the CNNlayer 531-1 closest to the input side, but only to subsequent layers;the CNN layer 531-2 and the CNN layer 531-3. Not limited thereto, thenumerical data #2 may be input only to CNN layer 531-1 closest to theinput side, but not to the subsequent layers; CNN layer 531-2 and CNNlayer 531-3. Alternatively, the numerical data #2 may be input to anyone CNN layer among the CNN layer 531-1, the CNN layer 531-2, and theCNN layer 531-3, but not to other CNN layers. Of course, the numericaldata #2 may be input to all of the CNN layer 531-1, the CNN layer 531-2,and the CNN layer 531-3.

In the above embodiment, the output of the machine learning model isassumed to output a medical image having no noise. However, the outputof the machine learning model, is not limited thereto.

For example, the machine learning model may output a segmentation resultof the medical image from the medical image and the numerical data. Forexample, in response to the input of a T1-weighted image or aT2-weighted image, an image in which an anatomical tissue or a lesionarea is divided is output as a segmentation result. The lesion area is,for example, an image area suspected of having cerebral infarction,ischemia, or cancer. The identification result of the medical image maybe output from the medical image and the numerical data. For example,for the input of a T1-weighted image, a T2-weighted image, or acombination of a T1-weighted image and a T2-weighted image, theprobability of being a cerebral infarction is output as anidentification result. The segmentation result of the medical image maybe output from the medical image and the numerical data. For example, inresponse to the input of the T1-weighted image, the T2-weighted image,or the combination of the T1-weighted image and the T2-weighted image,an image in which an image region having likelihood of cerebralinfarction is segmented is output as a segmentation result.

Even when training is performed based on medical images of differentimage types such as the T1-weighted images and the T2-weighted images,some of the parameters of the machine learning model are shared. Thus,the machine learning model can acquire versatility for inputting medicalimages of different image types while suppressing lowering of reductionin learning efficiency.

The machine learning model may output super resolution data of themedical data from the medical data and the numerical data. Superresolution data is medical data having a higher spatial resolution thaninput medical data. It is possible to generate the machine learningmodel by making DNN learned based on input data including medical dataand numerical data and super resolution data that is teacher data.

For example, when k-space data is acquired by a frequency-limitedhalf-Fourier method, the k-space data is partially lost and the spatialresolution is lowered. My making DNN based on input data includingk-space data and numerical data acquired by the half Fourier method andk-space data (teacher data) acquired from MR imaging without frequencylimitation, the machine learning model can be generated. By using themachine learning model, it is possible to output k-space data (superresolution data) in which a data deficient part is restored from k-spacedata acquired by the half Fourier method.

The machine learning model shown above can be generated by the modellearning apparatus 6 using supervised machine learning. The trainingsample is prepared by acquiring medical data under various acquisitionconditions. Specifically, the input data of the training sample includesinput medical data acquired under a certain acquisition condition, andnumerical data obtained by digitizing the acquisition condition. Theoutput data of the training sample includes output medical datacorresponding to the medical data and according to the purpose of themachine learning model. The output data is, for example, medical data inwhich noise is reduced as compared to input medical data if the purposeof the machine learning model is de-noising.

In the training function 612, the processing circuitry 61 applies inputmedical data and numerical data to a machine learning model to performforward propagation processing, and outputs output medical data. Next,the processing circuitry 61 applies the difference (error) between anestimated output medical data and the correct output medical data to themachine learning model to perform back propagation processing, andcalculates a gradient vector. Next, the processing circuitry 61 updatesparameters such as a weighting matrix and a bias of the machine learningmodel based on the gradient vector. By repeating the forward propagationprocessing, the back propagation processing, and parameter updatingprocessing while changing the training sample, a learned machinelearning model is generated.

As described above, the medical information processing apparatus 50includes the processing circuitry 51. The processing circuitry 51acquires medical data on a subject, acquires numerical data obtained bydigitizing an acquisition condition of the medical data, and applies amachine learning model 521 to input data including the numerical dataand the medical data, thereby generating output data based on themedical data.

According to the above configuration, the machine learning model 521uses, in addition to the medical data, numerical data obtained bydigitizing the acquisition condition corresponding to the medical data,and therefore, compared to the case where only the medical data isinput, the accuracy of output data can be improved. The number of theelements of the numerical data corresponds to the number ofpredetermined candidate conditions, and the value of each element is avalue according to whether or not the candidate condition in question isadopted. In other words, since the rules for digitizing the acquisitionconditions are predetermined, it is possible for the machine learningmodel to accurately determine the acquisition conditions at the time oflearning or application. This is also one factor to improve the accuracyof the output data.

According to at least one embodiment described above, the accuracy ofoutput data using machine learning can be improved.

The term “processor” used in the above description is intended to mean,for example, CPU, GPU, or circuits such as an application specificintegrated circuit (ASIC), a programmable logic apparatus (for example,a simple programmable logic device (SPLD) a complex Programmable Logicdevices (CPLD), and a field programmable gate arrays (FPGA). Theprocessor implements the respective functions 511 to 516 by reading andexecuting the program stored in the memory circuit. It should be notedthat the program may be directly incorporated in the circuit of theprocessor instead of storing the program in the memory circuit. In thiscase, the processor implements the respective functions 511 to 516 byreading and executing a program incorporated in the circuit. Further,instead of executing a program, each function 511 to 516 correspondingto the program may be realized by a combination of logic circuits. Itshould be noted that each processor according to the present embodimentis not limited to being configured as a single circuit for eachprocessor, and may be configured as one processor by combining aplurality of independent circuits to realize the function. Furthermore,the plurality of components in FIGS. 1, 3 and 7 may be integrated intoone processor to realize its function.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel embodiments described hereinmay be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the embodimentsdescribed herein may be made without departing from the spirit of theinventions. The accompanying claims and their equivalents are intendedto cover such forms or modifications as would fall within the scope andspirit of the inventions.

1. A medical information processing apparatus comprising a processingcircuitry, the processing circuitry being configured to acquire medicaldata on a subject, acquire numerical data obtained by digitizing anacquisition condition of the medical data, and apply a machine learningmodel to input data including the numerical data and the medical data togenerate output data based on the medical data.
 2. The medicalinformation processing apparatus according to claim 1, wherein themachine learning model receives inputs of the numerical data and themedical data and learns using teacher data.
 3. The medical informationprocessing apparatus according to claim 1, wherein the numerical dataincludes an element having two or more finite number of elements, thenumber of the elements corresponds to a number of a plurality ofcandidate conditions corresponding to the acquisition condition, and thevalue of the element corresponds to a numerical value according towhether or not the acquisition condition is adopted for the candidateconditions and/or a value.
 4. The medical information processingapparatus according to claim 3, wherein the value of the element has oneof a first value indicating that the acquisition condition is adoptedand a second value indicating that the acquisition condition is notadopted.
 5. The medical information processing apparatus according toclaim 3, wherein the numerical data includes an element having two ormore finite number of elements, and the elements by the number of theelements include a first element corresponding to the candidateconditions and a second element corresponding to none of the candidates.6. The medical information processing apparatus according to claim 1,wherein the acquisition condition includes a type of pulse sequence, aframe number, a type of k-space filling trajectory, a number of times ofrepetition of repetition operation and/or an acceleration factor ofparallel imaging.
 7. The medical information processing apparatusaccording to claim 1, wherein the processing circuitry generates data tobe provided for medical diagnosis on the subject as the output data. 8.The medical information processing apparatus according to claim 1,wherein the machine learning model is configured to multiply each of thenumerical data and the medical data by weight different and add aproduct for each channel.
 9. The medical information processingapparatus according to claim 1, wherein the processing circuitry isconfigured to acquire the numerical data obtained by digitizing theacquisition condition of medical imaging for the subject, acquire rawdata acquired by the medical imaging according to the acquisitioncondition as the medical data, and apply the machine learning model toinput data including the numerical data and the raw data or a medicalimage based on the raw data to generate the output data based on the rawdata or the medical image.
 10. The medical information processingapparatus according to claim 9, wherein the processing circuit performsreconstruction processing on the input data including the numerical dataand the raw data using a machine learning model to generate the imagedata on the subject.
 11. The medical information processing apparatusaccording to claim 9, wherein the acquisition condition is a number oftimes of acquisition and/or a direction of acquisition for each dataacquisition trajectory, and the numerical data is mask data in which anumerical value corresponding to the number of times of acquisitionand/or the direction of acquisition in target imaging is assigned to aplurality of data acquisition trajectory candidates having a finitenumber of elements.
 12. The medical information processing apparatusaccording to claim 11, wherein the processing circuitry determines thenumber of times of acquisition and/or the direction of acquisition forthe data acquisition trajectory candidates according to a user'sinstruction or automatically.
 13. The medical information processingapparatus according to claim 11, wherein the processing circuitrydetermines the number of times of acquisition and/or the direction ofacquisition for the data acquisition trajectory candidates according toa predetermined rule based on an imaging time in the medical imaging.14. The medical information processing apparatus according to claim 11,wherein the numerical value includes a first numerical value indicatingthat data acquisition is to be performed and a second numerical valueindicating that data acquisition is not to be performed.
 15. The medicalinformation processing apparatus according to claim 11, wherein each ofthe data acquisition trajectory candidates is a data acquisitiontrajectory in k-space related to the radial method, and linearly passessubstantially a center of the k-space.
 16. The medical informationprocessing apparatus according to claim 15, wherein an angular intervalbetween the data acquisition trajectory candidates is set to apredetermined angle.
 17. The medical information processing apparatusaccording to claim 15, wherein the data acquisition trajectorycandidates are arranged at substantially equal intervals at an anglebased on an angle obtained by dividing 360 degrees by the number of theelements.
 18. The medical information processing apparatus according toclaim 11, wherein each of the data acquisition trajectory candidates isa data acquisition trajectory in k-space related to a spiral method, andspirally passes through the k-space.
 19. The medical informationprocessing apparatus according to claim 11, wherein the processingcircuitry applies the machine learning model to the mask data and theraw data, generates de-noised raw data as the output data, and performsFourier transformation on the de-noised raw data to generate the medicalimage.
 20. The medical information processing apparatus according toclaim 11, wherein the processing circuitry performs Fouriertransformation on the raw data to generate a provisional medical image,applies the machine learning model to the mask data and the provisionalmedical image, and outputs the de-noised medical image as the outputdata.
 21. The medical information processing apparatus according toclaim 1, further comprising an imaging apparatus configured to performmagnetic resonance imaging on the subject and acquire k-space data asthe medical data.
 22. A medical information processing methodcomprising: acquiring medical data related to a subject; acquiringnumerical data obtained by digitizing an acquisition condition for themedical data; and applying a machine learning model to input dataincluding the numerical data and the medical data to generate outputdata based on the medical data.